Goto

Collaborating Authors

 San Cristóbal


RobuT: A Systematic Study of Table QA Robustness Against Human-Annotated Adversarial Perturbations

Zhao, Yilun, Zhao, Chen, Nan, Linyong, Qi, Zhenting, Zhang, Wenlin, Tang, Xiangru, Mi, Boyu, Radev, Dragomir

arXiv.org Artificial Intelligence

Despite significant progress having been made in question answering on tabular data (Table QA), it's unclear whether, and to what extent existing Table QA models are robust to task-specific perturbations, e.g., replacing key question entities or shuffling table columns. To systematically study the robustness of Table QA models, we propose a benchmark called RobuT, which builds upon existing Table QA datasets (WTQ, WikiSQL-Weak, and SQA) and includes human-annotated adversarial perturbations in terms of table header, table content, and question. Our results indicate that both state-of-the-art Table QA models and large language models (e.g., GPT-3) with few-shot learning falter in these adversarial sets. We propose to address this problem by using large language models to generate adversarial examples to enhance training, which significantly improves the robustness of Table QA models. Our data and code is publicly available at https://github.com/yilunzhao/RobuT.


Evo* 2022 -- Late-Breaking Abstracts Volume

Mora, A. M., Esparcia-Alcázar, A. I.

arXiv.org Artificial Intelligence

This volume contains the Late-Breaking Abstracts accepted at Evo* 2022 Conference, held in Madrid (Spain), from 20 to 22 of April. They were also presented as short talks as well as at the conference's poster session. The works present ongoing research and preliminary results investigating on the application of different approaches of Evolutionary Computation and other Nature-Inspired techniques to different problems, most of them real world ones. These are very promising contributions, since they outline some of the incoming advances and applications in the area of nature-inspired methods, mainly Evolutionary Algorithms.